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Abstract 


UTRPP addressed the i3 Development Grant Absolute Priority 1, Subpart 2: Increase equitable access to 
effective teachers or principals for low-income and high-need students. UTRPP infused comprehensive, 
job-embedded professional development (PD) for participating Residents and other teachers; 
established triads and quads to support Residents; and delivered a prescriptive focus on teaching and 
learning in urban settings with specialized emphasis on approaches established by the partner school 
district to ensure a seamless transition to induction and beyond. The PD built instructional capacity and 
improved both the Residents’ and Classroom Teachers’ content and pedagogical knowledge. The 
strength of UTRPP was in its unique approach, which included (a) targeting undergraduates for 
admission into the residency rather than graduate-level participants; (b) placing Residents in the field for 
an extended time, (c) integrating field experience with coursework aligned to trending issues in urban 
education; and (d) exposing Residents to culturally relevant content and field experiences unique to an 
urban setting (what UTRPP called a “place-conscious” approach to preparing educators in urban 
settings). The ultimate goal of UTRPP was to increase the academic achievement of low-income, high- 
need students in the partner district. 


UTRPP i3 Impact Report 


The Urban Teacher Residency Partnership Program (UTRPP) was a partnership between a large state 
university and large local school district in the Southeastern United States. The program created a 
pipeline from pre-service-to-practice in urban schools. It moved beyond traditional models of teacher 
preparation to a formalized extended residency program designed to change the staffing and support 
landscape for high-need schools. Unlike existing residency models that target graduate students, UTRPP 
began scaffolding field experiences with undergraduate elementary education majors. The model 
extended the length of pre-service, so the intensity and frequency of the residency experiences 
increased over a 2-year period. 


UTRPP provided pre-service teachers (named “residents”) with comprehensive pre-service to practice, 
job-embedded professional development (PD) experience with a focus on teaching and learning in urban 
settings, including: triads comprised of residents, collaborating teachers, and university supervisors for 
1*-year residents; quads comprised of residents, collaborating teachers (CTs), university supervisors, 
and university content coaches for 2"4-year residents; integrated university coursework and clinical 
experiences, differentiated PD for residents and all partnership school teachers, self-inquiry 
enhancement, and exposure to inclusive practices. Overall, the program was designed to improve 
content and pedagogical knowledge and ensure a seamless transition to induction in a high-needs, 
urban school setting. UTRPP residents attained priority hiring status, in the local school district that 
partnered with the university on the UTRPP program, upon successful completion of their second year in 
the program and all university degree requirements. Furthermore, UTRPP provided a model for other 
school districts that seek to increase and sustain the pool of teachers trained to meet the unique needs 
of students in urban settings and increase equitable access to effective teachers. 


UTRPP Goals 


UTRPP addressed the i3 Development Grant Absolute Priority 1, Subpart 2: Increase equitable access to 
effective teachers or principals for low-income and high-need students. UTRPP creates a pipeline from 
pre-service-to-practice in urban schools. It moved beyond traditional models of teacher preparation to a 
formalized extended residency program designed to change the staffing and support landscape for high- 
need schools. Unlike existing residency models that targeted graduate students, UTRPP began the 
scaffolding of field experiences with undergraduate elementary education majors. The model extended 
the length of pre-service, so the intensity and frequency of the residency experiences increase over a 2- 
year period. UTRPP infused comprehensive, job-embedded professional development (PD) for 
participating Residents and other teachers; established triads and quads to support Residents 
(Collaborating Teacher [CT], Partnership Resource Teacher, University Content Coach, and Resident); 
and delivered a prescriptive focus on teaching and learning in urban settings with specialized emphasis 
on approaches established by the partner school district to ensure a seamless transition to induction 
and beyond. The PD built instructional capacity and improved both the Residents’ and CTs’ content and 
pedagogical knowledge. 


Residents received coaching and support for 2 years in their established triad (first year) quad (final 
year), in addition to progressive, scaffolded field experiences. The inclusion of CTs and site-based staff in 
PD offerings further promoted the vital school-wide culture of learning necessary to tackle the obstacles 
presented by educating in urban partner school district settings. The content coaching quad 
configuration enhanced the customary pedagogical coaching that traditionally occurs in teacher 
preparation programs. Upon completion of the program, successful Residents had priority hiring status 
for full-time positions in the partner school district and received continued support. The CTs and other 


teachers had the opportunity to earn a Teacher Leader certification, which is a higher step on the career 
ladder for teachers in the partner school district and aligns with the USDOE’s policy goals of 
implementing Teacher Leader Pathways that allow teachers to advance in their field while remaining in 
the classroom. 


The strength of UTRPP was in its unique approach, which included (a) targeting undergraduates for 
admission into the residency rather than graduate-level participants; (b) placing Residents in the field for 
an extended time, (c) integrating field experience with coursework aligned to trending issues in urban 
education; and (d) exposing Residents to culturally relevant content and field experiences unique to an 
urban setting (what UTRPP called a “place-conscious” approach to preparing educators in urban 
settings). The ultimate goal of UTRPP was to increase the academic achievement of low-income, high- 
need students in the partner district. 


UTRPP Logic Model and Key UTRPP Activities 


Three broad goals framed UTRPP: (1) Increase the effectiveness of teachers serving low-income, high- 
need students; (2) Increase the equitable distribution of effective teachers for low-income, high-need 
students across schools; and (3) Increase the academic achievement of low-income, high-need students. 


The goals were supported by three key components (University Coursework, Residency Program, and 
university/partner school district Collaboration) with corresponding short, medium, and long-term 
outcomes. Therefore, all activities were part of an explicit plan designed to allow UTRPP to meet its 
stated goals and objectives. 


Implementation of UTRPP was focused on three key program components listed in the logic model: 

e University Coursework — Resident clinical experiences, 

e Residency Program — Established triads (Resident-CT-PRT) for first-year residents, quads 
(Resident-CT-PRT-Content Coach) for 2" year residents, and content-focus coaching for final- 
year residents; and 

e University/Partner School District Collaboration — Partnership schools, teacher leadership 
certificate, and UTRPP Advisory Board meetings. 


Threaded throughout these components were several key UTRPP features and activities. These items 
are described individually below. 


1. Integrated University Coursework and Clinical Experiences — Throughout the duration of the 
grant, coursework was designed and then annually revised based on evaluation feedback. 
These activities were done in partnership between the University and partner school district. 
Coursework consisted of explicit and purposeful connections to the clinical experiences 
residents engaged in within partnership schools. 

2. Content-Focused Coaching — All UTRPP residents engaged in extensive content-focused 
coaching. Coaching began in year one and expanded during year two. Residents had access to 
math, science, and literacy content coaches, and content coaches were also available to 
support Collaborating Teachers. 

3. Differentiated Collaborating Teacher Professional Development - Professional development 
for Collaborating Teachers was developed based on the needs of the UTRPP CTs broadly, as 
well as by individual schools specifically. Individualized professional development was offered 
based on teachers’ individual needs, and Collaborating Teachers received in-service credits to 


count towards recertification. 

4. Teacher Leader Certificate - In addition to professional development experiences designed 
for Collaborating Teachers, all Collaborating Teachers had access to participate in the 
University Teacher Leader Certification program. This program provides teachers the 
opportunity to cultivate leadership skills, while maintaining status as a classroom teacher. 

5. Cultivating Practice of Self-Inquiry - Throughout the duration of UTRPP all residents engaged 
in various experiences aimed at cultivating an inquiry stance. Residents in both year 1 and 
year 2 completed developmentally appropriate inquiry into their practice. Year 2 students 
continued to present at an annual Inquiry Conference, and Year 1 students presented their 
teacher inquiry within their cohort. Collaborating Teachers also conducted inquiry as part of 
their UTRPP professional development, as determined by the Principals at each site. 

6. Exposure to Inclusive Practices and Culturally-Relevant Content — Culturally relevant 
content and pedagogy was purposefully integrated into all UTRPP coursework and field 
experiences. Residents were immersed in cultivating a culturally relevant stance. 
Additionally, inclusive practices and culturally-relevant content/pedagogy was integrated into 
Coaching Teacher professional development experiences. 

7. Preparation-to-Practice Pipeline - Priority hiring status was extended from the partner 
school district to Residents who successfully completed UTRPP. 

8. UTRPP Advisory Board - Representatives from all stakeholders met on a regular basis to 
discuss UTRPP progress, challenges, and future implications. 


Project Evaluation Process and Results 


We worked closely with the partner school district and the University program managers to ensure that 
accurate UTRPP enrollment records were shared and up-to-date throughout the life of the 

project. Throughout the program, the evaluation team, the partner school district, and the University 
met annually to update measures, address challenges, and refine qualitative data collection protocols. 


Throughout the program, the evaluation process involved three distinct phases: 


1. UTRPP Implementation Fidelity Monitoring. This initial step in the measurement process 
determined whether the activities outlined by the UTRPP model were occurring as 
intended. Monitoring processes, including the acquisition of information such as coaching logs, 
professional development offering descriptions, observation data, and professional development 
attendance sheets, were implemented to collect evidence that funds were being utilized as 
proposed and to provide initial assessments of basic program fidelity. More details on this phase 
are provided in this report. 

2. Formative Evaluation Activities. Formative evaluation activities represented the next step in the 
measurement process. The information collected as part of formative evaluation activities 
provided more qualitative information about program facets than information collected as part of 
the monitoring processes. Surveys, interviews, and focus groups with UTRPP Residents, 
partnership school administrators, UTRPP alumni, and Collaborating Teachers provided valuable 
information about UTRPP experiences. Insights into successful program aspects or areas that 
needed change or improvement helped to establish program fidelity, and also assisted in 
improving the quality of the program deployment. 

3. Summative Evaluation Activities. Summative evaluation activities were the final step in the 
evaluation process. All summative evaluation work relied on quantitative district data that was 


shared with the external evaluator. The quantitative data were used to examine each of the 
project performance measures annually as well as to complete this impact report. 


Implementation Fidelity 


The implementation of the UTRPP i3 program was tracked and measured using the project Fidelity Matrix required by the evaluation guidelines 
for i3 grants. The matrix includes descriptors and indicators for the three key program components listed in the logic model: University 
Coursework, Residency Program, and University/Partner School District Collaboration. The matrix also specifies thresholds for adequate 
implementation, which vary depending on the program component. Per the matrix below, the following indicators were analyzed: resident 
clinical experiences, establishing triads for 1% year residents (not applicable in spring 2020 due to no 1* year residents), establishing quads for 2"? 
year residents, content-focused coaching for 2™ year residents, residents at partnership schools, and attendance at UTRPP Advisory Board 


Meetings. 
UTRPP i3 Fidelity Matrix 
Unit of Data Program-Level 
Indicators Definition & Data collection Score for Levels 8 7 
Implementation Sources Implementation 


Resident Clinical 
Experiences 


Establish Triads 


Resident attends 
clinical experiences 
each semester 


Resident 


Resident 


Clinical Log 


Hours 


Attendance 
information provided 
by Partnership 
Resource Teachers 
(PRTs)* 


Score based on % attendance at clinical 
experiences: 

0 (low) = attends < 80% of clinical 
experiences 

1 (moderate) = attends 80 - 89% of 
clinical experiences 

2 (high) = attends > 90% of clinical 
experiences 


residency 


Each 1° year Resident Resident is assigned to a triad 

for Residents in fi . . . Triad information . 

: Resident is assigned | Resident Placement . 0 (low) = Not assigned 
their 1° year of a CT and PRT ier provided by PRTs 

alas acs 

residency 2 (high) = assigned to quad 
Establish Quads | Each 2™ year Resident Resident Quad information Resident is assigned to a quad 
for Residents in | Resident is assigned Placement provided by PRTs 0 (low) = Not assigned 
their 2" year of | aCT, PRT, and List 


2 (high) = assigned to quad 


> 90% residents 
have adequate 
implementation 
score = 2 


Adequate 
Implementation 
score =6 
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multiple university 
Content Coaches 


Content- 
Focused 
Coaching for 
Residents in 
their 2" year of 
residency 


Each 2" year 
Resident 
participates ina 
cycle of each 
content area 
(science, math, and 
literacy) each 
semester 


Resident 


Coaching 
Cycle Log 


Coaching Cycle Log 
provided by PRTs 


0 (low) = < 3 cycles for the school year 


1 (moderate) = 4-5 cycles for the 
school year 


2 (high) > 6 cycles for the school year 


Partnership school score 


designee), and PRTs 


Assistant each month 


* Within UTRPP, PRTs have roles similar to traditional university supervisors. 


Partnershi eset par ticrsniP Partner School Resigent Resident Placement 
P school has residents | _. Placement ; 0 =no resident placed in school 
Schools District . list sent to Evaluators ; : 
(n=6) List 1 = resident placed in school 
Uiniverst Score based on % of organizations in 
. Y . : Attendance Records attendance at meetings. 
UTRPP Advisory | representatives, Meeting eoadedib 
. = 9 
Board Meetings | Partnership School UTRPP Attendance ace Ste 0 (low) = <74% 
Principals (or Roster Y 1 (moderate) = 75 - 84% 


2 (high) = > 85% 


Adequate 
Implementation = 
2 


UTRPP i3 Implementation Fidelity Results 2017, 2018 


The results in the table below represent the UTRPP i3 Fidelity Matrix Results for Program Years 3 (2017) and 4 (2018). For years 3 and 4 the 
following indicators were analyzed: resident clinical experiences, establishing triads for 1% year residents, establishing quads for 2"? year 
residents, content-focused coaching for 2™ year residents, residents at partnership schools, and attendance at UTRPP Advisory Board Meetings. 


During Program Year 3 (January 2017 to December 2017), UTRPP had low program implementation fidelity for University Coursework, and 
adequate implementation for the Residency Program and University/Partner School District Collaboration Components. In Program Year 4 
(January-December 2018), UTRPP had adequate program implementation fidelity for University Coursework and University/Partner School 


District Collaboration Components, and adequate implementation for the Residency Program component. 


Indicators 


Resident 
Clinical 
Experiences 


Establish Triads 
for Residents in 
their 1% year of 
residency 


Implementation 
Fidelity Score at 
Unit Level 


2017 =1 
2018 = 2 


2017 =2 
2018 = 2 


Unit Implementation Fidelity 
2017 


Moderate 
Overall, first-year residents and final- 
year residents (n=79)* completed 85% 
of expected clinical hours from January- 
December 2017 
e 42% (33/79) of all residents attended 
> 90% of clinical experiences 
e 53% (42/79) of all residents attended 
80 - 89% of clinical experiences 
e 5% (4/79) of all residents attended < 
80% of clinical experiences 


High 
100% of i3 15-year Residents were part 
of a triad 
(Resident, CT, and PRT) 
(24 Residents for January-May 2017) 


Unit Implementation Fidelity 
2018 


High 
Overall, first-year residents and 
final-year residents (n=64) *** 
completed 95% of expected clinical 
hours**** from January-December 
2018 
e 92% (59/64) of all residents 
attended > 90% of clinical 
experiences 
7% (5/64) of all residents attended 
80 - 89% of clinical experiences 


High 
100% of i3 1°t-year Residents were 
part of a triad 
(Resident, CT, and PRT) 
(18 Residents for January-May 2018) 


Program Level 
Thresholds 


> 90% residents have 
high implementation 
score = 2 


> 90% of residents with 
high implementation 
score = 6 


Program Level Fidelity 


2017 Low Program 
Implementation 


(score=1) 


2018 Adequate Program 
Implementation 


(score=2) 


2017 Adequate 
Implementation 


(score=6) 


(21 Residents for August-December 


(11 Residents for August-December 


2017) 2018) 
High 
High 6 
100% of i3 2"4-year Residents were part 100% of 13 2""-year Residents were 
i i - i Ww 
Establish 0 a aa p part of a quad 
uads for u 
Q ; : 2017 =2 ; (Resident, CT, PRT, Content 
Residents in (Resident, CT, PRT, Content Coaches) 
i mind 2018 =2 Coaches) 
their 2"° year (12 Residents for January-May 2017) : 
étrecidenc ; (21 Residents for January-May 2018) 
y (22 Residents for August-December . 
(14 Residents for August-December 
2017) 
2018) 
High 
100% of i3 2"¢-year Residents (n=21) 
participated in coaching cycles for 
Content- : each content area (science, math, 
High P 3 
Focused a and literacy) from January-April 
ae ; : 2 
Coaching for 2017 =2 pee ae yea sees 22) 2018 
‘ ‘ articipated in each content area 
Residents in 2018 =2 ilies 93% of i3 2"*-year Residents (n=14) 


their 2" year 
of residency 


(science, math, and literacy) from 
August-December 2017 


participated in coaching cycles for 
math and literacy ****** from 
August-December 2018 (Literacy 
14/14 and Math 13/14) 


2018 Adequate 
Implementation 


(score=6) 


Partnership 
Schools 


10 


2017=1 
2018 =1 


High 
Residents were placed in 100% (n=6) 
of the partnership schools from 
January to December 2017. 


High 
Residents were placed in 100% 
(January-April n=6, August- 
December n=5) of the 
partnership schools from 
January to December 2018. 


Low (0) = < 100% of 
partnership schools 
have a resident 
High (1) = 100% of 
partnership schools 
have a resident 


2017 Adequate 
Program 
Implementation 


(score=2) 


2018 Adequate 
Moderate Program 
: . . Implementation 
e All University representatives 
(faculty member, content coach, High (score=3) 
GA) were present at each meeting e All University 
(total = 15/15) representatives (faculty 
e Partner School District member, content coach, GA) 
representative was present for 4/5 were present at each 
meetings. meeting (total = 9/9) 
e 4/6 principals or designee were e Partner School District 
present at the January meeting, representative was present Low (0) = <75% 
5/6 principals were present for the for 2/3 meetings. organizations attend 
UTRPP February meeting, 3/6 principals e 4/6 principals or designee meetings 
Advisory or designee were present at the were present at the January Moderate (1) = 75 - 
paard 2017=1 April meeting, 3/6 principals or meeting, 5/6 principals were 849% organizations 
. 2018 =2 designee were present at the May present for the March : 
Meetings** . aos . er attend meetings 
meeting, and 4/6 principals or a meeting, 3/4 principals or 
designee were present at the designee were present at High (2) = 85% of 
November meeting (total = 19/30) the September meeting organizations attend 
e 2/3 PRTs were present in the (total = 12/16) ****** meeting 
January meeting, all PRTs were e All PRTs were present in the 
present for the February, April, January, March, and 
and May meetings (9/9), and 1/2 September meetings, (total 
PRTs were present at the = 6/6) 
November 2017 meeting (total = Overall, 29/34 (85.3%) of the 
12/14) representatives attended the 
Overall, 78% (50/64) of the Advisory Board Meetings. 
representatives attended the 
Advisory Board Meetings. 


* For January-April 2017, there were 36 total residents, and from August-December, there were 43 total residents. 


** There were five i3 UTRPP Advisory Board Meetings during Year 3 (January, February, April, May, and November 2017). 
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***For January-April 2018, there were 39 total residents, and from August-December 2018, there were 25 total residents. 
**** Based on school district policy, each resident was allocated four excused absent days a semester (32 hours). 

***** Science coaching does not occur in the Fall semester. 

There were three i3 UTRPP Advisory Board Meetings during Year 4 (January, March, and September 2018). 


****** Erom January-April there were six UTRPP schools, and from August-December there were 5 schools with two schools having the same 
principal 
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UTRPP i3 Implementation Fidelity Results 2019, 2020 


The results in the table below represent the UTRPP i3 Fidelity Matrix Results for Program Year 5 (January — December 2019) and Program Year 6 
(January - March 2020). For Program Years 5 and 6, the following indicators were analyzed: resident clinical experiences, establishing triads for 
1* year residents, establishing quads for 2™ year residents, content-focused coaching for 2™ year residents, residents at partnership schools, 
and attendance at UTRPP Advisory Board Meetings. 


Of note, is that Program Year 5 (2019), only had 2™ year residents during the fall 2019 semester. During Program Year 5 (January-December 
2019), UTRPP had adequate program implementation fidelity for University Coursework and University/Partner School District Collaboration 
Components, and adequate implementation for the Residency Program component. 


The 2020 data represent data collected in spring 2020 — this semester was the final semester for UTRPP and was supported through a no cost 
extension. Additionally, spring 2020 data represent adjusted data due to COVID-19. The week of March 23, 2020 e-learning began. During this 
time, residents participated in alternate clinical experiences and adjusted required hours from March 23, 2020 — April 17, 2020. During spring 
2020, UTRPP had adequate program implementation fidelity for University Coursework and University/Partner School District Collaboration 


Components, and adequate implementation for the Residency Program component. 


Indicators 


Resident 
Clinical 
Experiences 


Establish Triads 
for Residents in 


Implementation 
Fidelity Score at 
Unit Level 


2019 = 2 
2020 = 2 


2019 = 2 
2020 = N/A 


Unit Implementation Fidelity 
2019 


High 
Overall, first-year and final-year 
residents (n=34) * completed 95% of 
expected clinical hours** from January- 
December 2019 
e 92% (32/34) of all residents attended 
> 90% of clinical experiences 
e 6% (2/34) of all residents attended 80 
- 89% of clinical experiences 


High 


Unit Implementation Fidelity 
2020 


High 
Overall, all residents (n=10) * 
completed the expected clinical 
hours from January-April 2020. 


Program Level 
Thresholds 


> 90% residents have 
high implementation 
score = 2 


Program Level Fidelity 


2019 Adequate Program 
Implementation 


(score=2) 
2020 Adequate Program 
Implementation 


(score=2) 


2019 Adequate Program 
Implementation 
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their 1° year of 


100% of i3 1‘t-year Residents were part 


Not Applicable 


their 2" year 
of residency 


100% of i3 2"¢-year Residents (n=10) 
participated in coaching cycles for math 
and literacy **** from August-December 
2019. 


residency of a triad 
(Resident, CT, and PRT) 
(11 Residents for January-April 2019***) 
High High 
100% of i3 2"*-year Residents were 
Establish 100% of i3 2"*-year Residents were part at ae A a o 
Quads for of a quad P q 
2019 =2 (Resident, CT, PRT, Content 
Residents in (Resident, CT, PRT, Content Coaches) 
+ cand 2020 =2 ; j Coaches) 
their 2° year (14 Residents for January-April 2019) ao Residents for danuatyviareh 
: i uary- 
of residency (10 Residents for August-December 2020) 
2019) 
High High 
eee 100% of i3 2"-year Residents (n=14) 100% of i3 2"°-year Residents (n=10) 
Eaaiced participated in coaching cycles for each participated in coaching cycles for 
eoachine iar 2019 =2 content area (science, math, and literacy) each content area (science, math, 
a - oe from January-April 2019 and literacy) from January-March 


2020. 


> 90% of residents with 
high implementation 
score =6 


(score=6) 


2020 Adequate Program 
Implementation 


(score=4) 


Partnership 
Schools 


High 
Residents were placed in 100% (January- 
April n=4, August-December n=3) of the 
partnership schools from January to 
December 2019. 


High 
Residents were placed in 100% 


partnership schools from January — 
March 2020. 


Low (0) = < 100% of 
partnership schools have 
a resident 

High (1) = 100% of 
partnership schools have 
a resident 


2019 Adequate 
Program 
Implementation 


(score=2) 


14 


UTRPP 
Advisory 
Board 
Meetings** 


2019 =2 
2020 = 


High 

e All University representatives (faculty 
member, content coach, GA) were 
present at each meeting (total = 9/9) 

e Partner school district representative 
was present for 3/3 meetings. 

e 2/4 principals or designee were 
present at the January meeting, 2/4 
principals were present for the March 
meeting, 3/3 principals or designee 
were present at the November 
meeting (total = 7/11) ****** 

e All PRTs were present in the January, 
March, and November meetings, 
(total = 6/6) 

Overall, 25/29 (86.3%) of the 
representatives attended the Advisory 
Board Meetings. 


All University representatives 
(faculty member, content 
coach, GA) were present at the 
meetings. 

Partner School District 
representative was present for 
the meetings. 

3/3 principals or designee were 
present at the March meeting. 
All PRTs were present in the 
March meeting. 


Overall, 100% of the 
representatives attended the 
Advisory Board Meeting. 


Low (0) = <75% 
organizations attend 
meetings 

Moderate (1) = 75 - 84% 
organizations attend 
meetings 

High (2) => 85% of 
organizations attend 
meeting 


2020 Adequate 
Program 
Implementation 


(score=2) 


For January-April, there were 24 total residents that completed the residency hours, and from August-December, there were 10 total residents — 


all were 2™ year residents. 


** Based on school district policy, each resident was allocated four excused absent days a semester (32 hours). 


*** Eleven first-year residents began the program. One student withdrew. Data for this indicator is reported on the original eleven placements. 


**** Science coaching does not occur in the Fall semester. 


***** There were three i3 UTRPP Advisory Board Meetings during Year 5 (January, March, and November 2019). 


****** Erom January-April there were four UTRPP schools, and from August-December there were three schools. 


15 


Summary of Formative Evaluation Activities 


Throughout qualitative data collection residents noted that the increased field experiences provided a 
sense of preparedness for the classroom as driving factors for applying to UTRPP; however, residents 
consistently agreed that UTRPP was an intense and rigorous program. Whereas the increased field 
experiences and a sense of preparedness for the classroom were driving factors for applying to UTRPP, 
participants explicitly expressed that they felt “stressed,” “frustrated,” and “overwhelmed.” Many 
residents shared that they maintained outside jobs to manage their day-to-day finances and shared that 
the time dedicated to their jobs and families caused them to grapple with balancing coursework, 
residency, and daily life experiences. Whereas the residents recognized that their extended experiences 
in the classroom were potentially developing them into quality teachers, some residents openly shared 
they are questioning their choice of education as a profession. However, the majority of residents 
indicated that if they remain in teaching, they wished to teach in Title | settings and are thankful for the 
opportunity to impact their students as people and learners. 


Additionally, the qualitative evaluation process identified the following UTRPP items as important 
findings: 


1. Role of Consistent Coaching and Support — UTRPP is a clinically-centered teacher preparation 
program that immerses residents in authentic classroom practice from the onset of the program. 
Throughout these experiences the residents are afforded consistent coaching and support from 
various university and school-based teacher educators — PRTs, CTs, university faculty content 
coaches, and other partner school district personnel. Residents engaged in explicit coaching 
conversations weekly. These coached experiences emerged as a program component that 
residents and alumni perceived as extremely influential on their enactment of content and 
pedagogy. When asked to identify the aspects of UTRPP that were most influential on their growth 
as a teacher, residents and alumni all discussed how the ability to engage in on-going coaching 
enhanced their UTRPP experience. They perceived these coaching sessions as a space to gain 
knowledge, openly ask questions, and learn from experienced mentors. When reflecting on UTRPP 
coaching, alumni noted that the extensive coaching they received as a resident helped them to 
feel comfortable asking for assistance and collaborating with school-based coaches as a classroom 
teacher. 

2. Self-Efficacy and Confidence Entering the Classroom — A key finding is that all interviewed alumni 
communicated a sense of self-confidence and high levels of self-efficacy regarding their role as a 
classroom teacher. When asked how UTRPP influenced their confidence as a teacher, particularly 
as a teacher within a Title | context, interviewed alumni noted the importance of their extensive 
opportunities to engage in authentic teaching in authentic settings. They shared that learning to 
teach in a Title | context helped them to better understand the backgrounds, cultures, and 
experiences their students bring to the classroom. Whereas they recognized that UTRPP was 
intense and often overwhelming, they perceived their experiences to have successfully prepared 
them for the realities of teaching and they noted they felt more confident as early career teachers 
than their like peers. Another aspect of UTRPP that contributed to alumni confidence was the 
direct connection to the partner school district evaluation system. Alumni were appreciative of 
their knowledge of the partner school district rubric and the opportunity to be observed often 
using the rubric — sharing they now better understood the strong emphasis on this as a resident 
and that the evaluation system was not an intimidating process. 
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3. Challenges Encountered - One challenge encountered throughout the duration of the program 
was UTRPP’s inability to recruit the maximum number of allocated residents. The target number 
of 35 residents was not met in any implementation year, and only 41% of the of the original target 
number of 175 graduates from 2016-2020 was attained. UTRPP leadership attempted to enhance 
recruitment efforts, yet resident numbers remained low. Consistently, residents identified several 
challenges and stressors that may have added to low enrollment numbers. The UTRPP time 
commitment, and as a result, the financial strain weighed heavily on residents. Another emerging 
challenge connected to coherence and communication. A lesson learned throughout the program 
was the importance of coherence and communication across stakeholders and program 
requirements. Often residents discussed areas of confusion regarding coursework assignments 
and their alignment with partner school district expectations for teaching. Residents were 
extremely appreciative of assignments that directly connected to their experiences in the 
classroom, yet felt conflicted with assignments that did not seem to fit into the structure of their 
daily classroom experiences. Residents often felt that planning for the semester was difficult due 
to schedule changes and varying assignment expectations - residents shared how some 
assignments overlapped and that assignment due dates often conflicted. 


Communication with and across instructors and with Collaborating Teachers proved to provide 
additional challenges. Residents wondered if all Collaborating Teachers, and course instructors, 
truly understood their role and the tasks they were to complete. Communication is key for 
projects to be successful. Communication was challenging due to competing demands. Thus, 
having a set communication plan was necessary. The most successful UTRPP years were those 
when leadership and the Advisory Council, met regularly and consistently with shared agendas 
and action steps between meetings. As team members and program needs changed the 
communication systems needed to adapt. 


UTRRP Impact Evaluation 


Upon completion of the UTRPP Residency Program, district provided data were examined to determine 
if students taught by former UTRPP residents had different outcomes than students taught by teachers 
from traditional teacher preparation programs. Given the smaller than anticipated UTRPP cohorts and 
the large number of former residents that did not seek employment in the partner school district, as 
well as changes in achievement data that were available for kindergarten and first grade students, the 
methods originally planned for the impact evaluation were modified. Information on modifications are 
provided as they occurred. 


For this report we examined if students taught by former UTRPP residents during their first year in the 
partner school district differed from students taught by first year teachers from traditional teacher 
preparation programs on the following outcomes: reading achievement, mathematics achievement, 
number of days students attended school, and suspensions. Inclusion in this report was dependent on 
what grade the former UTRPP residents taught. For 1617 and 1718, pre and post achievement data were 
available for grades 2 —5 and in 1819 pre and post achievement data were available for grades K—5. 
Because achievement data were not available for kindergarten or first grade in some years, as was 
originally proposed, this limited the number of former residents that were included. For example, 
although six former UTRPP residents were employed by the partner school district during the 1617 AY, 
only two, who both taught third grade, are included in the impact analyses — the other four taught 
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kindergarten and first grade. For the 1718 AY, two of the six former residents who were employed by 
the partner school district were also excluded from this analysis because they taught kindergarten. All 
four former residents working in the partner school district in 1819 were included in this analysis. 


A total of 10 former UTRPP residents are included in these analyses. Two from 1617 (both who taught 
third grade), four from 1718 (one who taught second grade, one who taught third grade, and two who 
taught fourth grade), and four from 1819 (three who taught kindergarten and one who taught fifth 
grade). All of the former residents taught at Title | schools and were matched with first year teachers at 
Title | schools who were trained through traditional teacher preparation programs. Given the small 
number of residents to find matched classrooms for and the lack of teacher level data that was originally 
listed to use in a propensity score matching process, former residents and traditionally trained first year 
teachers were manually matched based on “baseline” reading and mathematics achievement data. In 
1617 and 1718, Stanford-10 (for grades 2 and 3) and FCAT (for grades 4 and 5) achievement data from 
the prior spring were used as baseline data. In 1819, i-Ready data were used so the 2018 fall 
assessments were used as baseline data. All matched teachers had baseline assessment scores within 5 
points of each other. Because different assessments were used for different grades and across the 
years, all achievement data were converted to z-scores by academic year, test type, and grade level. We 
also examined if students differed in their attendance, measured as days attended, and suspensions, 
measured as 0 days of suspension vs. 1 or more days of suspension. The SURVEYREG and 
SURVEYLOGISTIC procedures in SAS were used to account for the clustering of students within teachers. 


Results 


Based on the regression models, no statistically significant differences were found in any of the 
outcomes. Results from the four regression models are provided in Table 1. In the results, the primary 
coefficient of interest is for the variable, UTRPP — those regression coefficients represent the differences 
in each outcome for students taught by former UTRPP residents and those taught by teachers from 
traditional teacher preparation programs. Remember, because the reading and mathematics 
achievement data were transformed to z-scores, the regression coefficients from those models are to be 
interpreted as differences in standardized means. 


Table 1 

Regression Analyses Results Examining Differences in Outcomes between Students Taught by 
Former UTRPP Residents and Those Taught by Teachers from Traditional Teacher Prep 
Programs 


Estimate Standard t value p-value 
Error 
Reading Achievement 
Intercept 103.78 5.66 18.33 <.0001 
UTRPP 1.93 7.94 0.24 0.8081 
Mathematics Achievement 
Intercept -0.08 0.07 -1.16 0.2485 
UTRPP 0.04 0.11 0.36 0.7184 
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Days of School Attended 
Intercept 103.78 5.66 18.33 <.0001 


UTRPP 1.93 7.94 0.24 0.8081 


Suspended from School** 


Intercept 2.72 0.25 -10.93 <.0001 


UTRPP -0.67 0.40 -1.68 0.0931 


Notes: ** Results are from logistic regression model predicting suspended = 1 
Conclusion 


Although UTRPP had good implementation fidelity, given the low numbers of students who participated 
in the program and the low number of UTRPP residents that went on to work as teachers in the partner 
school district, it was not surprising to see no evidence in student outcomes. At the end of the funded i3 
program the partner school district was planning to reimagine a more nimble and innovative program, 
based on the lessons learned during the current project, however, since then, the University that 
sponsored UTRPP has since closed its College of Education and will no longer be training teacher 
educators. Perhaps the partner school district can take some of the more successful practices (including 
content coaching and integrated elementary STEM lessons) and include them in their beginning teacher 
support and retention programs. 
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